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Relevance of the Hierarchical Linear Model to TIMSS Data Analyses 

Abstract 

Multilevel international data have been released from the Third International 
Mathematics and Science Study (TIMSS), which provides an opportunity to apply multilevel 
modeling techniques in educational research. In this article, TIMSS factors are classified in 
fixed and random categories according to the project design. Relevant methods for the 
multilevel data analyses have been discussed in light of the fundamental relationship between the 
traditional dummy variable regression and the hierarchical linear modeling. 
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Relevance of the Hierarchical Linear Model to TIMSS Data Analyses 

With more than 40 nations participating and five grade levels assessed in two subject 

areas, the Third International Mathematics and Science Study (TIMSS) is the largest and most 

comprehensive project in comparative education (Martin & Kelly, 1996). Secondary analyses of 

the TIMSS data may enrich information on the U.S. condition of education in a cross-nation 

context (Peak, 1996). Because TIMSS data have a hierarchical structure with students nested in 

schools and schools nested in countries , some researchers attempted to adopt the hierarchical 

linear model (HLM) for international comparisons. Based on the eighth grade science data, 

researchers of the TIMSS International Center reported, “An unconditional, three level analysis 

of science achievement indicates that the variance can be approportioned 26% within school, 

35% between schools, [and] 39% between countries” (Gregory & Shen, 1999, p. 8). Meanwhile, 

Gregory and Shen (1999) cautioned that the lack of randomization at the country level may 

not need the variance partitioning in HLM. Since the choice of research methodology inevitably 

affects results of TIMSS data analyses, the purpose of this study is to examine features of the 

TIMSS design, and specify proper fixed and random factors within the hierarchical data 

structure. This investigation may help clarify relevance of HLM for TIMSS investigations. 

Fixed and Random Factors in Hierarchical Modeling 

Multilevel data can be analyzed by several software packages, such as HLM, MlwiN, 

and SAS (Wang, 1997). Accordingly, the terminology varies across the different computer 

software applications. Bryk and Raudenbush (1992) acknowledged in their book for HLM that. 

The models discussed in this book appear in diverse literatures under a variety of titles. 

In sociological research, they are often referred to as multilevel linear models [cf 
Goldstein, 1987; Mason et al., 1983]. In biometric applications, the terms mixed-effects 
models and random-effects models are common [cf Elston & Grizzle, 1962; Laird & 
Ware, 1982]. (p. 3) 
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At any levels of the hierarchy, factors can be either fixed or random, depending on the research 

design. Casella and Berger (1990) defined: 

A factor is a fixed factor if all the values of interest are included in the experiment. 

A factor is a random factor if the values of interest are not included in the experiment and 
those that are can be considered to be randomly chosen from the values of interest, (p. 

529) 

Milliken and Johnson (1984) added that “a model is called a mixed or mixed effects model if 
some of the factors in the treatment structure are fixed effects and some are random effects” (p. 
213). 

In TIMSS, research interest is confined in around 40 participating nations. Thus, country 
can be treated as a fixed factor, and the research findings are not designed for generalization over 
all countries in the world. On the other hand, schools and students are sampled randomly 
within each nation. Factors at these levels can be treated as random factors, and the sample 
statistics will be used to represent the condition of education in each nation. Therefore, the 
hierarchical model contains both fixed and random factors. The HLM computing, as a method of 
mixed-effects modeling, should be relevant to the TIMSS data analyses. 

Dummy Factor Regression 

Before TIMSS, the International Association for the Evaluation of Educational 
Achievement (lEA) sponsored the first and second round of international mathematics and 
science studies. International comparisons were made using a dummy variable approach in the 
regression analyses (personal communication with John Keeves, April 13, 1998). For simplicity, 
a two level model is illustrated in this proposal to articulate relations between the dummy 
variable regression and the hierarchical linear modeling. According to Bryk and Raudenbush 
(1992), the hierarchical model can be presented as: 

Level 1 (e.g., school) yij=poj+PijXy+rij 
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Level 2 (e.g., country) Poj^Yoo'h'oiWj+Uoj 

Pij=yio^iiWj+u,j (p. 29) 

Besides a random component (r^) at the school level, the inclusion of country-level variations 
(e.g., Uoj and Ujj) implied existence of randomization from the country sampling. By 
substituting Poj=yoo'h'oiWj-HUoj and Pij=yio-h'iiWj+u,j into yij=Poj+PijXij+ry, the hierarchical model 
can be further combined in a single equation: 

yij=yoo^ioXij+/oi Wj+/„XijWj+Uoj+u,jX ij+r^ (p. 29) 

Although a more complex error term (Upj+u,jX jj+rjj) has been introduced in the hierarchical 
model, the lack of random sampling at the country level has led some researchers to question 
the partitioning of variance at the country level (Gregory & Shen, 1999). Because TEMSS 
researchers may not be interested in generalizing the results to non-participating nations, the 
cross-nation differences can described by fixed dummy factors (e.g., Wj) with Uoj=u,j=0. In this 
case, the hierarchical model is equivalent to a regression equation with dummy factors: 

yij=yoo^ioXij+/oi Wj+/i ,XijWj+Uoj+u,jX y+ry 

where rjj ~ N (0, a^). 

Bryk and Raudenbush (1992) concurred that “if Upj and u,j were null for every j”, the 
combined HLM model is “equivalent to an OLS [Ordinary Least Square] regression model” (p. 
15). Therefore, if fixed factors at the country level are coded by dummy variables, the dummy 
variable regression is relevant to the hierarchical data analyses in TIMSS. 

The differentiation between fixed and random effects can be further delineated by format 
of the null hypothesis. For instance, due to differences in educational policy and practice, each 
country has a unique school system. With many schools randomly surveyed in each nation, it 
would make sense to make inference about the average national performance in TIMSS. Thus, 
the null hypothesis at the country level is regarding the mean score comparison: 
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Hq: ^, = ^2=••• = ^n 
for n countries participating in TIMSS. 

Within each country, since a random sample of schools were selected in TIMSS to 

represent its condition of education, the school level variables can be specified as random 

factors affecting student learning experience. Consider a two-level specification between 

student and school, the HLM can be written as: 

Level 1 (e.g., student) yy=Poj+PijXij+rij 

Level 2 (e.g., school) Poj=Yoo-^oiWj+Uoj 

Pij=Yio^nWj+u,j 

Model in combined form: 
yy^Yoo^ioXy^oi ,XjjWj+Uoj+u,jX y+ry 

Unlike the aforementioned levels of country-school hierarchy, Uoj and u,j should be retained in 
the model to reflect the involvement of random school sampling. The corresponding null 
hypothesis in this case is: 

^0" ^ school ^ 

Thus, statistical null hypotheses may differ across the levels of hierarchical structure, 
depending on a proper specification of the fixed and random effects. Whether random sampling 
has been incorporated at the national level does not affect the relevancy of HLM to the TIMSS 
data analyses. Instead, dummy variable regressions can be adopted along with the random 
coefficient modeling to specify the mixed structure of fixed and random effects in TIMSS. The 
clarification of research methodology may facilitate analyses of TIMSS data in a multilevel 
context. 
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